{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "max_pooling_in_cnn.ipynb",
      "provenance": [],
      "authorship_tag": "ABX9TyNSn7tCxdd5drPz21kTt1Fy",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/SummerLife/EmbeddedSystem/blob/master/MachineLearning/gist/max_pooling_in_cnn.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "60YV_QQlAylb",
        "colab_type": "text"
      },
      "source": [
        "## Introducing Max Pooling\n",
        "\n",
        "Max pooling is a type of operation that is typically added to CNNs following individual convolutional layers.\n",
        "\n",
        "When added to a model, max pooling reduces the dimensionality of images by reducing the number of pixels in the output from the previous convolutional layer.\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "hHj-hymHFXKm",
        "colab_type": "text"
      },
      "source": [
        "## Why Use Max Pooling?\n",
        "\n",
        "There are a couple of reasons why adding max pooling to our network may be helpful.\n",
        "\n",
        "### Reducing Computational Load\n",
        "Since max pooling is reducing the resolution of the given output of a convolutional layer, the network will be looking at larger areas of the image at a time going forward, which reduces the amount of parameters in the network and consequently reduces computational load.\n",
        "\n",
        "### Reducing Overfitting\n",
        "Additionally, max pooling may also help to reduce overfitting. The intuition for why max pooling works is that, for a particular image, our network will be looking to extract some particular features.\n",
        "\n",
        "Maybe, it’s trying to identify numbers from the MNIST dataset, and so it’s looking for edges, and curves, and circles, and such. From the output of the convolutional layer, we can think of the higher valued pixels as being the ones that are the most activated.\n",
        "\n",
        "With max pooling, as we’re going over each region from the convolutional output, we’re able to pick out the most activated pixels and preserve these high values going forward while discarding the lower valued pixels that are not as activated."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "5QCBp5yS_pMh",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "import keras\n",
        "from keras.models import Sequential\n",
        "from keras.layers import Activation\n",
        "from keras.layers.core import Dense, Flatten\n",
        "from keras.layers.convolutional import *\n",
        "from keras.layers.pooling import *"
      ],
      "execution_count": 3,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "hlJoLb40F6Yl",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "model_valid = Sequential([\n",
        "    Dense(16, input_shape=(20,20,3), activation='relu'),\n",
        "    Conv2D(32, kernel_size=(3,3), activation='relu', padding='same'),\n",
        "    MaxPooling2D(pool_size=(2, 2), strides=2, padding='valid'),\n",
        "    Conv2D(64, kernel_size=(5,5), activation='relu', padding='same'),\n",
        "    Flatten(),\n",
        "    Dense(2, activation='softmax')\n",
        "])"
      ],
      "execution_count": 5,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Qfk_hptrF8jH",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 357
        },
        "outputId": "acb56c05-13a3-46a9-d45e-133c3b3fd67f"
      },
      "source": [
        "model_valid.summary()"
      ],
      "execution_count": 6,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Model: \"sequential_1\"\n",
            "_________________________________________________________________\n",
            "Layer (type)                 Output Shape              Param #   \n",
            "=================================================================\n",
            "dense_1 (Dense)              (None, 20, 20, 16)        64        \n",
            "_________________________________________________________________\n",
            "conv2d_1 (Conv2D)            (None, 20, 20, 32)        4640      \n",
            "_________________________________________________________________\n",
            "max_pooling2d_1 (MaxPooling2 (None, 10, 10, 32)        0         \n",
            "_________________________________________________________________\n",
            "conv2d_2 (Conv2D)            (None, 10, 10, 64)        51264     \n",
            "_________________________________________________________________\n",
            "flatten_1 (Flatten)          (None, 6400)              0         \n",
            "_________________________________________________________________\n",
            "dense_2 (Dense)              (None, 2)                 12802     \n",
            "=================================================================\n",
            "Total params: 68,770\n",
            "Trainable params: 68,770\n",
            "Non-trainable params: 0\n",
            "_________________________________________________________________\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "kI6S05qYG4WL",
        "colab_type": "text"
      },
      "source": [
        "Once we go down to the max pooling layer, we see the value of the dimensions has been cut in half to become 10 x 10. This is because, as we saw with our earlier examples, a filter of size 2 x 2 along with a stride of 2 for our max pooling layer will reduce the dimensions of our input by a factor of two, so that’s exactly what we see here."
      ]
    }
  ]
}